A batch framework can typically support a single task that represents a job. The framework can also execute serial tasks with the assistance of a batch journal; however, there may not be other dependency support. Additionally, tasks execution can require connecting a client and manually launching a batch engine. Other batch engines can employ a workflow style where a task that finishes executing includes the logic about what to do next.
For example, the following limitations can exist in current batch processing frameworks. The framework requires a client for processing batch jobs, and the batch cannot simply be submitted with the assumption that the job will be executed, as the job may not run if a client is not open. Moreover, the framework may not support parallelism. There can be various scenarios where a batch job that processes transactions, for example, could be broken into separate sub-jobs and executed against different batch server instances to increase throughput and reduce overall execution time. The batch framework may not allow the creation of a dependency tree for the different tasks that comprise the batch job.
When a batch task is executing, successful execution can fail due to infrastructure failures, including power outages, server crashes or connectivity issues, among other failures. When this occurs the batch engine tries to re-execute the failing task. Ultimately, the outcome of the task is desired to be the same no matter what happens to the infrastructure.